Skip to content

fix(ci): drop byte-identity gates that fail on dependency bumps - #176

Merged
auto-wood merged 2 commits into
mainfrom
fix/ci-drop-nonfunctional-artifact-gates
Nov 16, 2026
Merged

auto-wood merged 2 commits into
mainfrom
fix/ci-drop-nonfunctional-artifact-gates

Conversation

@auto-wood

Copy link
Copy Markdown
Contributor

Root cause

The required xcframework check compared the committed archives (crates/spoke-connect/bindings/swift/xcframework/spoke_connectFFI.xcframework/*.a) byte-for-byte against a fresh build. Any Cargo.lock bump that touches spoke-connect's dependency tree changes the compiled archives, so dependency-only PRs fail with drift detected without any behavior change — #171 (uuid 1.29.0 → 1.29.2) and #172 (sha2 0.11.0 → 0.11.1).

The same class of snapshot coverage existed in three other places: bindgen regeneration byte diffs, generated-tree git diff plus a generated-Rust size bound, and C++ committed-provenance hash verification.

Change

Remove coverage that asserts previously committed bytes, sizes or record hashes instead of behavior:

  • xcframework: committed-vs-rebuilt archive manifest comparison (the whole two-tree mode)
  • bindings: Swift/Python uniffi regeneration byte diff
  • codegen: git diff --exit-code on generated trees and the generated Rust size bound
  • C++: committed provenance hash verification in both carrier lanes

Kept as behavior coverage: C ABI symbol checks, C++/Kotlin/C#/Swift smokes, golden vectors, schema count, opaque-JSON type contract, lockstep version surfaces. The xcframework transfer manifest remains; apply-xcframework-artifact.sh still checksum-verifies the downloaded bundle before staging.

Effect on open dependency PRs

After merge, rebasing #171/#172/#173/#175 onto main drops these gates from their runs. A remaining failure on those branches would be a real behavior or compatibility failure (e.g. the TypeScript 7 major bump), not artifact byte drift.

Verification

  • node --test tooling/connect/cpp-build.test.mjs tooling/connect/apply-xcframework-artifact.test.mjs — 22 pass
  • pnpm run codegen + schema count + opaque-JSON + deep-ref-hoist test — pass; generated trees unchanged
  • cargo +nightly check --workspace — pass
  • workflow YAML parses; node tooling/docs/twin-parity.mjs — OK

@greptile-apps

greptile-apps Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge within its stated scope of removing committed-byte checks.

Summary

Removes committed-byte comparisons from native builds, binding generation, and schema generation.

  • Swift builds no longer have to match the committed bundle byte for byte.
  • C++ carrier checks no longer compare committed files with recorded hashes.
  • Code generation no longer requires an unchanged generated tree or a fixed Rust file size.
  • Swift and Python bindings no longer need to match fresh generated bytes in CI.

auto-wood explicitly describes the removed byte, size, and recorded-hash checks as intentional. No separate actionable regression was found.

Reviews (1) · Last reviewed commit: "fix(ci): drop byte-identity gates that f..." · Reviewed by Greptile

greptile-apps[bot]
greptile-apps Bot previously approved these changes Nov 16, 2026
@m-inspector

m-inspector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

✅ Verdict: ship it · Score 94% (mergeable)

Findings Count
🔴 must-fix 0
🟠 should-fix 0
🟡 may-fix 1
🔵 nit 0

📋 Review

What this PR does: This PR removes validation that asserts previously committed artifact bytes instead of behavior: the xcframework committed-vs-rebuilt archive comparison (the whole two-tree mode), uniffi Swift/Python regeneration byte diffs, codegen git diff on generated trees plus the generated-Rust size bound, and C++ committed provenance hash verification in both carrier lanes. The stated root cause is that dependency-only Cargo.lock bumps (e.g. uuid 1.29.0→1.29.2, sha2 0.11.0→0.11.1) rebuild behaviorally identical archives and fail the required xcframework gate with drift detected. Behavior coverage is kept: C ABI symbol checks, C++/Kotlin/C#/Swift smokes, golden vectors, schema count, opaque-JSON type contract, lockstep version surfaces, and the apply script's transfer-manifest verification before staging.

Findings

🟡 Xcframework refresh text now claims builds are never bit-reproducible, contradicting the repo's same-input reproducibility contract

  • Evidence:

    [DOCS-01] Blanket non-reproducibility claim contradicts the documented same-input check

    • Evidence: CONTRIBUTING.md:124 — the edited closing sentence now reads "Builds are not bit-reproducible across runs; the transfer manifest and producer record identify the applied bytes."; it replaced "A fresh build may differ from a later rebuild; retain the actual two-tree comparison and investigate drift rather than treating successful application as reproducibility" (pr.diff, hunk @@ -121,13 +121,13 @@).
    • Evidence: .agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:52 — "Independent reproducibility requires another real source build with equivalent inputs."
    • Evidence: .agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:58 — "For a focused reproducibility check, retain the first output, clean the task-owned build directory, and compile again at the exact same path with the same inputs. Compare complete raw outputs …"; :66 — "Independent cold builds and whole-file mismatch controls establish reproducibility within their declared input boundary."
    • Impact: The maintainer guide now states as fact that bit-reproducibility is unattainable, while the repo's tooling-decision record prescribes exactly the two-build, same-path comparison as the way to establish it and still tells a native comparison to distinguish changed inputs from same-input nondeterminism (…native-artifacts.md:78). The removed clause was the only CONTRIBUTING statement keeping that comparison/drift-investigation in the refresh path, so a maintainer following this file is steered away from the check the knowledge record requires (and from treating a byte-identical control as evidence).
    • Attack result (three-way): counter-example partly holds — separate CI runs use different absolute output paths, so LC_ID_DYLIB/UUID/section offsets normally differ and CHANGELOG.md:541 already gates Windows freshness on build revisi…
  • Location: CONTRIBUTING.md:124

  • Category: docs

  • Merge class: may-fix

Revalidated findings

Reassessed 8/9 prior finding(s) this round · 0 invalid-output · 1 omitted · 0 over cap · 9 recheckable · context coverage: truncated

finding disposition evidence thread
CONTRIBUTING still advertises git diff on generated trees in verify-codegen addressed (verified-fix) current-code -
Seat coverage declaration (not a finding) dismissed (non-fix-dismissal) current-code -
Committed xcframework bundle now has no verifier against its committed manifest/provenance sidecar unverifiable (no-evidence) - -
verify-codegen no longer asserts the committed generated trees match a fresh regeneration dismissed (non-fix-dismissal) current-code -
Codegen knowledge still asserts a zero-drift proof covers generated bytes addressed (verified-fix) current-code -
cpp-build.mjs --verify removed, leaving committed C carrier hashes unenforced while docs still demand they match dismissed (non-fix-dismissal) current-code -
Committed C/C++ carrier provenance hashes are no longer verified anywhere dismissed (non-fix-dismissal) current-code -
xcframework refresh instructions still require a two-tree comparison whose tool was deleted addressed (verified-fix) current-code -
verify-codegen no longer detects committed generated-tree drift dismissed (non-fix-dismissal) current-code -

Linked-issue AC

none — the PR links no issue (issues is empty; #171/#172/#173/#175 appear only as open dependency PRs expected to be rebased, not as linked acceptance criteria).

✅ Verified

- CONTRIBUTING.md:124 was read directly: the refresh paragraph now closes with "Builds are not bit-reproducible across runs; the transfer manifest and producer record identify the applied bytes."
- The replaced clause was confirmed from pr.diff hunk @@ -121,13 +121,13 @@, which shows the removed "retain the actual two-tree comparison and investigate drift" sentence.
- .agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md:52,58,66 still require an independent source build with equivalent inputs and a complete raw-output comparison at the same path, and :78 still lists "distinguish changed inputs from same-input nondeterminism" as a when-to-apply condition.
- verify-codegen row checked against package.json:15: the codegen script chains generated code with assert-schema-count.mjs, assert-opaque-json-types.mjs, the deep-ref-hoist node test and cargo check --workspace; no generated-tree git diff gate exists under the worktree, so the row correction is accurate.
- C-carrier refresh sentence checked against .github/workflows/cpp-connect.yml:156-168 and tooling/connect/cpp-build.mjs:298-312: the Windows lane uploads the provenance.json the build rewrote, so the replacement text is accurate.
- Prior-finding revalidation: three findings resolved as verified-fix (CONTRIBUTING git diff advertisement, codegen zero-drift knowledge assertion, xcframework two-tree refresh instructions); four candidates dismissed as non-fix.
- ❓ PR-body verification claims (22 tests across cpp-build.test.mjs and apply-xcframework-artifact.test.mjs, pnpm run codegen + schema/opaque/deep-ref-hoist, cargo +nightly check --workspace, twin-parity.mjs) are not evidenced by the review document.

🗑️ Considered & rejected

- verify-codegen no longer asserts a fresh regeneration matches the committed generated trees — dismissed as non-fix: the removed snapshot comparison is the exact gate class this PR intends to drop, and it fails on dependency-only bumps without behavior change.
- Codegen knowledge still asserted that a zero-drift proof covers generated bytes — revalidated as addressed (verified-fix), so no finding was raised.
- cpp-build.mjs --verify removed while maintainer docs still demand committed C carrier hashes match — dismissed as non-fix: the documented refresh path sources provenance.json from the lane build that rewrote it (cpp-connect.yml:156-168), so docs and scripts agree.
- Committed C/C++ carrier provenance hashes are no longer verified anywhere — dismissed as non-fix, same reasoning: transfer identity comes from the refreshed producer record plus the committed carrier ABI/smoke checks, not a stale hash.
- xcframework refresh instructions requiring a two-tree comparison whose tooling was deleted — revalidated as addressed at this head.
- Seat coverage declaration — dismissed as a non-finding (process note, not a defect in the diff).
- Omission of the deep-ref-hoist unit-test step from the verify-codegen documentation row — dropped as non-admissible polish.
- CHANGELOG.md:541 gating Windows freshness on build revision rather than byte equality was considered as support for the new wording and partly accepted: separate runs use different absolute output paths, so the blanket claim is defensible for cross-run artifacts; only the unqualified quantifier remains a defect.

🛠️ Plan to fix

Expand fix plan
Only DOCS-01 (`may-fix`, XS, LOW risk) remains. No code, CI or artifact behavior depends on the wording.

1. **Edit `CONTRIBUTING.md:124`** (the closing sentence of "Refreshing the Swift xcframework"): replace the blanket claim with a qualified one, e.g. "Artifacts from separate CI runs are not byte-identical — different absolute build paths change `LC_ID_DYLIB`, UUIDs and section offsets — so the transfer manifest and producer record identify the applied bytes. To establish reproducibility, rebuild at the same path with identical inputs and compare complete raw outputs." Keep the manifest + producer-record identity sentence; keep the pointer to the same-input two-tree comparison.
2. **Confirm parity with the knowledge record** in `.agents/knowledge/tooling-decisions/ci-assembled-committed-native-artifacts.md` § "Compare independent builds with identical complete inputs" (`:52-66`) and the `:78` when-to-apply entry; the new wording must not read as retiring the same-input obligation. Do not edit that file — it is already correct.
3. **Check the adjacent `:46` statement** ("apply a validated refresh without comparing two build trees") still reads as application identity, not as a reproducibility claim, so the two sentences do not conflict.
4. **Verification gates**: re-read the refresh section for self-consistency with `CHANGELOG.md:541` (build-revision freshness, not byte equality); run `node tooling/docs/twin-parity.mjs` only if the touched section is mirrored in the docs site. No build/test matrix is required — the change is prose inside a maintainer doc.

Review 2 · commit 7b29e0c · since 74650c6

@m-inspector m-inspector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mstar-inspector line comments · round 1 · 74650c6

Comment thread tooling/connect/cpp-build.mjs Resolved
Comment thread package.json Resolved
Comment thread .github/workflows/cpp-connect.yml Resolved
Comment thread CONTRIBUTING.md Resolved
@greptile-apps
greptile-apps Bot dismissed their stale review November 16, 2026 09:21

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@m-inspector m-inspector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mstar-inspector line comments · round 2 · 7b29e0c

Comment thread CONTRIBUTING.md Resolved
@auto-wood
auto-wood merged commit 105f4a3 into main Nov 16, 2026
25 checks passed
@auto-wood
auto-wood deleted the fix/ci-drop-nonfunctional-artifact-gates branch November 16, 2026 11:07
@auto-wood

Copy link
Copy Markdown
Contributor Author

All six review threads resolved.

  • Thread 2 (committed generated-tree drift) — fixed in fix(ci): verify committed codegen output instead of a regenerated worktree #177. verify-codegen no longer runs codegen first; it now checks the committed trees: schema-count and opaque-JSON asserts, deep-ref-hoist test, @42ch/spoke-schemas typecheck, then cargo check --workspace. Regeneration is the author step. This removes the defect where CI compiled a freshly rewritten worktree instead of the trees that publish.
  • Threads 1 and 3 (C carrier provenance hashes) — removal is the intended contract. The docs now state it: provenance.json hashes are build audit data, not a CI gate; committed carriers are covered by the C ABI symbol check and the C++ smoke. CONTRIBUTING.md no longer requires recorded hashes to match committed bytes.
  • Threads 4, 5, 6 — docs corrected. No two-tree comparison instruction, no zero-drift claim, and reproducibility is scoped to identical inputs (source, lockfile, toolchain, target, output path).

Verification of the original failure class: #171 (uuid) and #172 (sha2) were updated onto post-#176 main. Both now pass the required xcframework check (#171 24m00s, #172 50m29s); both failed that check before on archive byte drift. All required checks are green on both PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants